Convex Learning with Invariances
نویسندگان
چکیده
Incorporating invariances into a learning algorithm is a common problem in machine learning. We provide a convex formulation which can deal with arbitrary loss functions and arbitrary losses. In addition, it is a drop-in replacement for most optimization algorithms for kernels, including solvers of the SVMStruct family. The advantage of our setting is that it relies on column generation instead of modifying the underlying optimization problem directly.
منابع مشابه
Learning with Invariance via Linear Functionals on Reproducing Kernel Hilbert Space
Incorporating invariance information is important for many learning problems. To exploit invariances, most existing methods resort to approximations that either lead to expensive optimization problems such as semi-definite programming, or rely on separation oracles to retain tractability. Some methods further limit the space of functions and settle for non-convex models. In this paper, we propo...
متن کاملConvex Invariance Learning
Invariance and representation learning are important precursors to modeling and classification tools particularly for non-Euclidean spaces such as images, strings and nonvectorial data. This article proposes a method for learning invariances in data while jointly estimating a model. The technique results in a convex programming problem with a consistent and unique solution. Representation varia...
متن کاملParametrizing filters of a CNN with a GAN
It is commonly agreed that the use of relevant invariances as a good statistical bias is important in machine-learning. However, most approaches that explicitly incorporate invariances into a model architecture only make use of very simple transformations, such as translations and rotations. Hence, there is a need for methods to model and extract richer transformations that capture much higherl...
متن کاملSemi-Supervised Learning in Reproducing Kernel Hilbert Spaces Using Local Invariances
We propose a framework for semi-supervised learning in reproducing kernel Hilbert spaces using local invariances that explicitly characterize the behavior of the target function around both labeled and unlabeled data instances. Such invariances include: invariance to small changes to the data instances, invariance to averaging across a small neighbourhood around data instances, and invariance t...
متن کاملLearning Spatio-Temporal Invariances
We present a neural network model for the unsupervised learning of high order visual invariances. The model is demonstrated on the problem of estimating sub-pixel stereo disparity from a temporal sequence of unprocessed image pairs. After learning on a given image sequence, the model's ability to detect sub-pixel disparity generalises, without additional learning, to image pairs from other sequ...
متن کامل